Broad match control

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on a computer-readable storage medium, for evaluating a creative. A method includes: identifying one or more selection keywords that are associated with a creative, the selection keywords used in determining when the creative is shown in response to received content requests that include one or more request keywords; identifying matching criteria for comparing the selection keywords and the request keywords, the matching criteria defined by a range from an exact match to a broad match and including a matching threshold in the range; establishing a bid modifier associated with the matching criteria, the bid modifier modifying a bid associated with the creative based on an extent of a match determined between the selection keywords and the request keywords for a given content request; and evaluating the creative responsive to received content requests based on the matching criteria and the bid modifier.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of, and claims priorityto, U.S. patent application Ser. No. 14/361,637, filed May 29, 2014,which application is a National Stage Application under 35 U.S.C. § 371and claims the benefit of International Application No.PCT/SE2013/050924, filed Jul. 22, 2013. The disclosure of the foregoingapplications are hereby incorporated by reference in their entirety.

BACKGROUND

This specification relates to information presentation.

The Internet provides access to a wide variety of resources. Forexample, video and/or audio files, as well as web pages for particularsubjects or particular news articles, are accessible over the Internet.Access to these resources presents opportunities for other content(e.g., advertisements) to be provided with the resources. For example, aweb page can include slots in which content can be presented. Theseslots can be defined in the web page or defined for presentation with aweb page, for example, along with search results.

Slots can be allocated to content sponsors through a reservation systemor an auction. For example, content sponsors can provide bids specifyingamounts that the sponsors are respectively willing to pay forpresentation of their content. In turn, a reservation can be made or anauction can be performed, and the slots can be allocated to sponsorsaccording, among other things, to their bids and/or the relevance of thesponsored content to content presented on a page hosting the slot or arequest that is received for the sponsored content.

SUMMARY

In general, one innovative aspect of the subject matter described inthis specification can be implemented in methods that include a methodfor evaluating a creative. The method comprises: identifying one or moreselection keywords that are associated with a creative, the selectionkeywords used in determining when the creative is shown in response toreceived content requests that include one or more request keywords;identifying matching criteria for comparing the selection keywords andthe request keywords, the matching criteria defined by a range from anexact match to a broad match, and wherein the matching criteria includesa matching threshold in the range; establishing a bid modifier that isassociated with the matching criteria, the bid modifier modifying a bidassociated with the creative based on an extent of a match that isdetermined between the selection keywords and the request keywords for agiven content request; and evaluating the creative responsive toreceived content requests based at least in part on the matchingcriteria and the bid modifier.

These and other implementations can each optionally include one or moreof the following features. Identifying the matching criteria can includesuggesting a matching criteria for a content sponsor associated with thecreative. Identifying the matching criteria can include enablingselection of matching criteria from an entire portion of the range by acontent sponsor associated with the creative. Enabling selection ofmatching criteria can include presenting an interface to a contentsponsor associated with the creative. The interface can include acontrol for specifying matching criteria in the range. The control canbe a sliding scale. Establishing the bid modifier can includeautomatically selecting the bid modifier for a content sponsorassociated with the creative. Establishing the bid modifier can includeenabling selection of the bid modifier by a content sponsor associatedwith the creative. Enabling selection of the bid modifier can includepresenting an interface to the content sponsor associated with thecreative. The interface can include a control for adjusting the bidmodifier over the range. The control can enable selection of differentbid modifiers for different portions of the range. The control canenable selection of a function to define an adjustment of the bidmodifier over the range. A selection of matching criteria in the rangecan be received. A selection of a bid modifier to be associated with thereceived selection of the matching criteria can be received. A pluralityof bid modifiers to be applied to different matching criteria over therange can be established. One of the plurality of bid modifiers can beapplied for a given request based on an extent of match between theselection keywords and request keywords for the given request. A requestfor content can be received where the request includes one or morerequest keywords. The request keywords can be compared to the selectionkeywords associated with the creative. An extent of a match can bedetermined based on the comparison. A determination can be made that thematching criteria is satisfied based on the extent and, when so, the bidmodifier can be used in evaluating whether to serve the creativeresponsive to the request. The creative can be an advertisement.

In general, another aspect of the subject matter described in thisspecification can be implemented in computer program products. Acomputer program product tangibly embodied in a computer-readablestorage device comprises instructions. The instructions, when executedby a processor, cause the processor to: identify one or more selectionkeywords that are associated with a creative, the selection keywordsused in determining when the creative is shown in response to receivedcontent requests that include one or more request keywords; identifymatching criteria for comparing the selection keywords and the requestkeywords, the matching criteria defined by a range from an exact matchto a broad match and including a matching threshold in the range,wherein the identifying includes presenting an interface to a contentsponsor associated with the creative, the interface including a firstcontrol for specifying the matching threshold; establish a bid modifierthat is associated with the matching criteria, the bid modifiermodifying a bid associated with the creative based on an extent of amatch that is determined between the selection keywords and the requestkeywords for a given content request, wherein the establishing includesenabling selection of the bid modifier by the content sponsor using asecond control included in the interface; and evaluate the creativeresponsive to received content requests based at least in part on thematching criteria and the bid modifier.

These and other implementations can each optionally include one or moreof the following features. Identifying the matching criteria can includeproviding a suggestion for the matching criteria to the content sponsor.The first control can be a sliding scale. Establishing the bid modifiercan include automatically selecting the bid modifier for the contentsponsor. The second control can enable selection of different bidmodifiers for different portions of the range. The second control canenable selection of a function to define an adjustment of the bidmodifier over the range. The creative can be an advertisement.

In general, another aspect of the subject matter described in thisspecification can be implemented in systems. A system comprises one ormore processors and one or more memory elements including instructions.The instructions, when executed, cause the one or more processors to:identify one or more selection keywords that are associated with acreative, the selection keywords used in determining when the creativeis shown in response to received content requests that include one ormore request keywords; identify matching criteria for comparing theselection keywords and the request keywords, the matching criteriadefined by a range from an exact match to a broad match, and wherein thematching criteria includes a matching threshold in the range; establisha bid modifier that is associated with the matching criteria, the bidmodifier modifying a bid associated with the creative based on an extentof a match that is determined between the selection keywords and therequest keywords for a given content request; and evaluate the creativeresponsive to received content requests based at least in part on thematching criteria and the bid modifier.

These and other implementations can each optionally include one or moreof the following features. Identifying the matching criteria can includesuggesting a matching criteria for a content sponsor associated with thecreative. Identifying the matching criteria can include enablingselection of matching criteria from an entire portion of the range by acontent sponsor associated with the creative. Enabling selection ofmatching criteria can include presenting an interface to a contentsponsor associated with the creative. The interface can include acontrol for specifying matching criteria in the range. The control canbe a sliding scale. Establishing the bid modifier can includeautomatically selecting the bid modifier for a content sponsorassociated with the creative. Establishing the bid modifier can includeenabling selection of the bid modifier by a content sponsor associatedwith the creative. Enabling selection of the bid modifier can includepresenting an interface to the content sponsor associated with thecreative. The interface can include a control for adjusting the bidmodifier over the range.

Particular implementations may realize none, one or more of thefollowing advantages. A content sponsor can, for a content item,configure matching criteria and corresponding bids at various levels ofbroadness. A content sponsor can associate multiple bids with aselection keyword for a content item at various levels of broadnesswhile specifying the selection keyword once rather than multiple timesfor the content item.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment for providingcontent to a user.

FIG. 2 is a flowchart of an example process for evaluating a creative.

FIG. 3A illustrates an example campaign management user interface.

FIG. 3B illustrates example user interface controls.

FIG. 4 is a flowchart of an example process for evaluating a creative.

FIG. 5 is a block diagram of computing devices that may be used toimplement the systems and methods described in this document, as eithera client or as a server or plurality of servers.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

One or more selection criteria (e.g., keywords) associated with acreative can be used in determining when the creative is shown inresponse to requests for content. The requests for content can includeone or more request criteria (e.g., or request keywords). A contentsponsor can identify matching criteria for comparing the selectioncriteria (e.g., keywords) and the request criteria (e.g., keywords),such as by designating a matching threshold in a range defined, forexample, from an exact match to a broad match. For example, the contentsponsor can use a slider control or some other type of control todesignate the matching threshold. The content sponsor can also establish(e.g., define) a bid modifier that is associated with the matchingcriteria designation, where the bid modifier modifies a bid associatedwith the creative based on a designated modifier and an extent of amatch that is determined between the selection criteria and the requestcriteria for a given request. A content management system, in responseto a request for content, can evaluate the creative based at least inpart on the matching criteria and the bid modifier, to determine whetherthe creative is selected to be served in response to the request.

FIG. 1 is a block diagram of an example environment 100 for providingcontent to a user. The example environment 100 includes a network 102,such as a local area network (LAN), a wide area network (WAN), theInternet, or a combination thereof. The network 102 connects websites104, user devices 106, content providers 108, publishers, and a contentmanagement system 110. The example environment 100 may include manythousands of websites 104, user devices 106, and content providers 108.The content management system 110 may be used for selecting andproviding content in response to requests for content. The contentproviders 108 can be, for example, advertisers. Other types of contentproviders are possible.

A website 104 includes one or more resources 105 associated with adomain name and hosted by one or more servers. An example website 104 isa collection of web pages formatted in hypertext markup language (HTML)that can contain text, images, multimedia content, and programmingelements, such as scripts. Each website 104 can be maintained by acontent publisher, which is an entity that controls, manages and/or ownsthe website 104.

A resource 105 can be any data that can be provided over the network102. A resource 105 can be identified by a resource address that isassociated with the resource 105. Resources 105 include HTML pages, wordprocessing documents, portable document format (PDF) documents, images,video, and news feed sources, to name only a few. The resources 105 caninclude content, such as words, phrases, videos, images and sounds, thatmay include embedded information (such as meta-information hyperlinks)and/or embedded instructions (such as scripts).

A user device 106 is an electronic device that is under control of auser and is capable of requesting and receiving resources 105 over thenetwork 102. Example user devices 106 include personal computers, tabletcomputers, mobile communication devices (e.g., smartphones),televisions, set top boxes, personal digital assistants and otherdevices that can send and receive data over the network 102. A userdevice 106 typically includes one or more user applications, such as aweb browser, to facilitate the sending and receiving of data over thenetwork 102. The web browser can interact with various types of webapplications, such as a game, a map application, or an e-mailapplication, to name a few examples.

A user device 106 can request resources 105 from a website 104. In turn,data representing the resource 105 can be provided to the user device106 for presentation by the user device 106. A request for content canbe received by the content management system 110 from a user device 106,such as in association with a request for a resource 105. The contentmanagement system 110 can select one or more content items to provide inresponse to the request. For example, the request for content caninclude characteristics (e.g., size, supported media types) of one ormore content “slots” that are defined for the requested resource 105 ora search results page. As another example, resource keywords associatedwith a requested resource 105 or a search query 117 can be provided tothe content management system 110.

Content items selected by the content management system 110 can includecontent items having characteristics or selection criteria matching thecharacteristics of content slots and that are identified as relevant tospecified request criteria (such as resource keywords or search queries117). For example, content items having keywords that match the resourcekeywords or the search query 117 can be selected by the contentmanagement system 110. The content management system 110 can also selectcontent items at least in part based on the results of an auction. Thecontent management system 110 can provide one or more selected contentitems to the requesting user device 106, for presentation on the userdevice 106.

In some implementations, the content management system 110 can selectcontent items based at least in part on results of an auction. Forexample, content providers 108 can provide bids specifying amounts thatthe content providers 108 are respectively willing to pay forpresentation of their content items. In turn, an auction can beperformed and the slots can be allocated to content providers 108according, among other things, to their bids and/or the relevance of acontent item to content presented on a page hosting the slot or arequest that is received for the content item. For example, when a slotis being allocated in an auction, the slot can be allocated to thecontent provider 108 that provided the highest bid or a highest auctionscore (e.g., a score that is computed as a function of a bid and/or aquality measure). When multiple slots are allocated in a single auction,the slots can be allocated to a set of bidders that provided the highestbids or have the highest auction scores.

In some implementations, some content providers 108 prefer that thenumber of impressions allocated to their content and the price paid forthe number of impressions be more predictable than the predictabilityprovided by an auction. For example, a content provider 108 can increasethe likelihood that its content receives a desired or specified numberof impressions, for example, by entering into an agreement with apublisher 109, where the agreement requires the publisher 109 to provideat least a threshold number of impressions (e.g., 1,000 impressions) fora particular content item provided by the content provider 108 over aspecified period (e.g., one week). In turn, the content provider 108,publisher 109, or both parties can provide data to the contentmanagement system 110 that enables the content management system 110 tofacilitate satisfaction of the agreement.

For example, the content provider 108 can upload a content item andauthorize the content management system 110 to provide the content itemin response to requests for content corresponding to the website 104 ofthe publisher 109. Similarly, the publisher 109 can provide the contentmanagement system 110 with data representing the specified time periodas well as the threshold number of impressions that the publisher 109has agreed to allocate to the content item over the specified timeperiod. Over time, the content management system 110 can select contentitems based at least in part on a goal of allocating at least a minimumnumber of impressions to a content item in order to satisfy a deliverygoal for the content item during a specified period of time.

A content provider 108 or content sponsor can create a content campaignassociated with one or more content items using tools provided by thecontent management system 110. For example, the content managementsystem 110 can provide one or more account management user interfacesfor creating and managing content campaigns. The account management userinterfaces can be made available to the content provider 108, forexample, either through an online interface provided by the contentmanagement system 110 or as an account management software applicationinstalled and executed locally at a content provider's client device.

A content provider 108 can, using the account management userinterfaces, provide campaign parameters 120 which define a contentcampaign. The content campaign can be created and activated for thecontent provider 108 according to the parameters 120 specified by thecontent provider 108. The campaign parameters 120 can be stored in aparameters data store 122. Campaign parameters 120 can include, forexample, a campaign name, a preferred content network for placingcontent, a budget for the campaign, start and end dates for thecampaign, a schedule for content placements, content (e.g., creatives),bids, and selection criteria. Selection criteria can include, forexample, a language, one or more geographical locations or websites, andone or more selection terms.

The parameters 120 can also include matching criteria for comparing theselection criteria (e.g., keywords) to request criteria (e.g., keywords)associated with a request for content. In some implementations, aselection keyword can be associated with one or match types. Match typescan include, for example, exact match, near-exact match, phrase match,near-phrase match, broad match modifier, and broad match. Other matchtypes are possible. A request keyword can match a broad match keyword,for example, if the request keyword includes some or all of the wordsincluded in the broad match keyword, in any order. In someimplementations, the request keyword can also include words other thanthe words included in the broad match keyword. In some implementations,a request keyword can also match a broad match keyword if the requestkeyword includes singular or plural forms, synonyms, and other relevantvariations of one or more words included in the broad match keyword. Abroad match modifier is a modifier that can be applied to a wordincluded in a broad match keyword to indicate that for a match to occurthe word needs to be included in the request keywords as, for example,an exact match or a close variant of the word, where close variantsinclude misspellings, singular/plural forms, abbreviations and acronyms,and stemmings.

In some implementations, a request keyword can match a phrase matchkeyword if the request keyword includes at least the phrase matchkeyword as an exact phrase, with the words of the phrase in the requestkeyword being in the same order as the words in the phrase matchkeyword. In some implementations, the request keyword can match thephrase match keyword if the request keyword includes words other thanthe phrase match keyword. A near-phrase match keyword is similar to aphrase match keyword with a difference being that with a near-phrasematch keyword, a match between the request keyword and the near-phrasematch keyword can occur, for example, if the request keyword includes atleast the near-phrase match keyword as an exact phrase or as a closevariant of the exact phrase. In some implementations, a request keywordcan match an exact match keyword if the request keyword exactly matchesthe exact match keyword and does not include any other words other thanthe exact match keyword. A near-exact match keyword is similar to anexact match keyword except that with a near-exact match keyword a matchcan occur, for example, if the request keyword is a close variant of thenear-exact keyword.

The options for various match types can be sufficient for some or allcontent providers 108. However, to define desired matching criteria,some content providers may be required to enter selection keywordsmultiple times with a different match type and bid for each instance ofthe selection keyword. If a campaign includes a large number ofselection keywords, creating and maintaining multiple instances of someor all selection keywords can be difficult and time consuming. Asanother example, some content providers 108 can desire to have morerefined control for matching a request keyword to a selection keywordthan is offered by a set of discrete match types, such as forfine-tuning selection of a matching criteria to provide a desired returnof investment. In addition, some content providers 108 may wish to varytheir bids based on the match type without necessarily any other changeto the selection criteria.

In some implementations, for example, a matching criteria can beidentified (e.g., defined) by a content provider 108 by a range from anexact match to a broad match. Defining the matching criteria caninclude, for example, designating a matching threshold in the range. Forexample, the account management user interface can include a userinterface element (e.g., a sliding scale) for specifying matchingcriteria in a range. As described in more detail below, the parameters120 can also include a bid modifier for each selection keyword, where abid modifier is associated with a matching criteria designation andmodifies a bid associated with a creative based on a modifier and anextent of a match that is determined between the selection keywordassociated with the bid modifier and request keywords associated with agiven request.

FIG. 2 is a flowchart of an example process 200 for evaluating acreative. The process 200 can be performed, for example, by the contentmanagement system 110 described above with respect to FIG. 1.

One or more selection keywords that are associated with a creative areidentified (202). The selection keywords can be used in determining whenthe creative is shown in response to requests that are received forcontent where the requests for content include one or more requestkeywords. The creative can be, for example, an advertisement or someother type of content. The selection keywords can be identified, forexample, in response to a content sponsor specifying the selectionkeywords on a campaign management user interface.

Matching criteria for comparing the selection keywords and the requestkeywords are identified (204). For example, the matching criteria can bedefined by a range, for example, from an exact match to a broad match.Identifying the matching criteria can include, for example, receiving adesignation of a matching threshold in the range. For example, one ormore user interface controls can be presented to a content sponsorassociated with the creative on the campaign management user interface.The user interface controls can include, for example, a sliding scalefor specifying matching criteria in the range.

For example, FIG. 3A illustrates an example campaign management userinterface 300. The user interface 300 can be included, for example, inone or more user interfaces that a user, such as a campaign sponsor, canuse to configure a campaign. The sponsor can select a tab 302 to displaya campaign configuration area 304. The sponsor can view a list 306 ofcampaigns by selecting a control 308. The sponsor can edit an existingcampaign in the campaign configuration area 304 by selecting the name ofan existing campaign (e.g., a name 310) in the campaign list 306. Thesponsor can select a content item for the campaign using a control 312.Selection criteria and a bid for the selected content item can bespecified using controls 314 and 316, respectively.

The sponsor can use a slider control 318 to configure matching criteriafor the entered selection criteria. The matching criteria can be definedby a range, for example, from an exact match to a broad match. Thesponsor can select and move a control 320 to a position on the slidercontrol 318 to designate a matching threshold in the range. Whenevaluating received requests for content, a content management systemcan compare request keywords to the selection criteria specified by thesponsor, and can identify the content item as a candidate content itemif the request keywords match the selection criteria at the level ofexactness indicated by the control 320 or, in some implementations, at amore exact level.

Other types of controls can be used to define matching criteria. Forexample, FIG. 3B illustrates an example group 350 of user interfacecontrols that includes a dial control 352 that defines a range from anexact match to a broad match. The sponsor can adjust the dial control352 to designate a matching threshold in the range. In someimplementations, when first configuring particular selection criteria, asuggestion 354 for a matching criteria setting can be displayed to thesponsor. The sponsor can either accept or adjust the suggested setting.The suggestion 354 can be determined, for example, from historicalperformance information for the sponsor or from sponsors similar to thesponsor (e.g., peer content providers in a same vertical as thesponsor). Performance information can also be presented to a user uponselection. That is, in order to ensure that a proper level is selected,performance at a given level can be presented to the user so as toenable for an adjustment to be made as required.

As another example, as shown for a slider control 356, match type labels358 a-f can be displayed at various points of the range defined by theslider control 356, to aid the sponsor in selecting a particularthreshold. For example, the labels 358 a-f and corresponding points onthe slider control 356 correspond to match types of exact match,near-exact match, phrase match, near-phrase match, broad match modifier,and broad match, respectively. The sponsor may be familiar with suchmatch types and can select a position on the slider control 356 based onan understanding of the match types. The sponsor can move a control 359to a particular point corresponding to one of the labels 358 a-f or to aposition on the slider control 356 that is between two of the points 358a-f. When positioned between the two discrete levels, a modified orcustom match can be defined, for example, based on a percentage of matchor based on other intermediary criteria between the two fixed levels.

In some implementations, in addition to or instead of the labels 358a-f, an example selection keyword can be displayed along with examplerequest keywords that would match the selection keyword at variouslevels of broadness. In some implementations, other information can bedisplayed that is related to a current setting of the control 259. Forexample, predicted statistics such as impressions, conversions, costs,and other information can be displayed that are estimated based on acurrent match threshold corresponding to the control 259. The predictedstatistics can change as the user moves the control 259.

Referring again to FIG. 2, a bid modifier that is associated with thematching criteria designation is established (206). The bid modifiermodifies a bid associated with the creative based on an extent of amatch that is determined between the selection keywords and the requestkeywords for a given request. The bid modifier can be defined, forexample, with information received from a content sponsor through a userinterface.

For example, as shown in FIG. 3B, a group 360 of user interface controlscan be presented to the sponsor. Bid modifier fields 364 a-d can bedisplayed along with a slider control 365. The sponsor can enter a bidmodifier into a respective bid modifier field 364 a-d, to configure amodification to a bid entered using a bid control 366. A particular bidmodifier can be applied, for example, if a selection keyword matches arequest keyword at a level of exactness corresponding to the bidmodifier. For example, a bid modifier of 0.6 (e.g., corresponding to thebid modifier field 364 d) can be applied to the bid of ten entered usingthe bid control 366 (e.g., creating a modified bid of six) if theselection keyword matches the request keyword at a level of exactnesscorresponding to a control 368 positioned on the slider 365 (e.g., anexactness level slightly less than broad match). In someimplementations, the sponsor can enter one or more specific, particularbids (e.g., rather than bid modifier(s)), corresponding to one or morelevels of exactness.

In some implementations, default bid modifiers are presented in the bidmodifier fields 364 a-d and the sponsor can either accept or adjust thedefault bid modifiers. In some implementations, such as corresponding toan advanced operation mode, the sponsor is enabled to select or define afunction which defines bid modifier values over the matching criteriarange. For example, the sponsor can select a control 370, which canresult in the display of a function selection/definition interface (notshown in FIG. 3B).

Referring again to FIG. 2, the creative is evaluated responsive toreceived content requests based at least in part on the matchingcriteria and the bid modifier (208). For example, the creative can beevaluated as described below in process 400.

FIG. 4 is a flowchart of an example process 400 for evaluating acreative. The process 400 can be performed, for example, by the contentmanagement system 110 described above with respect to FIG. 1.

A request for content including one or more request keywords is received(402). For example, a request for content for one or more content slotscan be received from a user device. The request keywords can, forexample, be keywords associated with a resource from whence the requestoriginated or with a search query.

The request keywords are compared to selection keywords associated withthe creative (404) and an extent of a match is determined based on thecomparison (406). For example, in some implementations, the extent ofmatch is a determined semantic-relevance score which estimates asemantic difference between the request keywords and the selectionkeywords. In some implementations, machine-learning systems and scoringmodels are used to determine the extent of match. In someimplementations, an ROI (Return on Investment)-based score is used todetermine the extent of match, such as by comparing a predictedinteraction rate (e.g., predicted click through rate (PCTR)) associatedwith the request keywords to a predicted interaction rate associatedwith the selection keywords. In some implementations, a keyword matchscore based on a model trained using human evaluators can be used todetermine the extent of match, where the keyword match score indicateshow closely a request keyword is to a particular selection keyword.

A determination is made that the matching criteria is satisfied based onthe extent (408). For example, the matching criteria can be defined by arange from an exact match to a broad match and a sponsor-providedmatching threshold in the range can be identified. The matchingthreshold can be, for example, a numerical value within the range ofzero to one hundred, where zero represents an exact match and onehundred represents a broad match. For instance, an examplesponsor-provided matching threshold can be forty percent. The extent ofmatch can be compared to the matching threshold. If the extent of matchis equal to or greater than the matching threshold, a determination canbe made that the matching criteria is satisfied. For example, thematching criteria can be satisfied when the extent of match is forty onepercent and the matching threshold is forty percent.

A bid modifier is used in evaluating whether to serve the creativeresponsive to the request (410). For example, a bid modifier associatedwith the extent of match can be determined and can be applied to a bidassociated with the creative. For example, the bid can be ten and thedetermined bid modifier can be 0.8. In such an example, application ofthe bid modifier to the bid can result in a modified bid of eight. Anauction can be performed with the creative and other creatives, with themodified bid being used for the creative. The creative can be servedresponsive to the request, for example, if the creative has a highestbid or a highest auction score (e.g., a score that is computed as afunction of a bid and/or a quality measure or other factors).

FIG. 5 is a block diagram of computing devices 500, 550 that may be usedto implement the systems and methods described in this document, aseither a client or as a server or plurality of servers. Computing device500 is intended to represent various forms of digital computers, such aslaptops, desktops, workstations, personal digital assistants, servers,blade servers, mainframes, and other appropriate computers. Computingdevice 550 is intended to represent various forms of mobile devices,such as personal digital assistants, cellular telephones, smartphones,and other similar computing devices. The components shown here, theirconnections and relationships, and their functions, are meant to beillustrative only, and are not meant to limit implementations of theinventions described and/or claimed in this document.

Computing device 500 includes a processor 502, memory 504, a storagedevice 506, a high-speed interface 508 connecting to memory 504 andhigh-speed expansion ports 510, and a low speed interface 512 connectingto low speed bus 514 and storage device 506. Each of the components 502,504, 506, 508, 510, and 512, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 502 can process instructions for executionwithin the computing device 500, including instructions stored in thememory 504 or on the storage device 506 to display graphical informationfor a GUI on an external input/output device, such as display 516coupled to high speed interface 508. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices500 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 504 stores information within the computing device 500. Inone implementation, the memory 504 is a computer-readable medium. Thecomputer-readable medium is not a propagating signal. In oneimplementation, the memory 504 is a volatile memory unit or units. Inanother implementation, the memory 504 is a non-volatile memory unit orunits.

The storage device 506 is capable of providing mass storage for thecomputing device 500. In one implementation, the storage device 506 is acomputer-readable medium. In various different implementations, thestorage device 506 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device, a flash memory or other similarsolid state memory device, or an array of devices, including devices ina storage area network or other configurations. In one implementation, acomputer program product is tangibly embodied in an information carrier.The computer program product contains instructions that, when executed,perform one or more methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 504, the storage device 506, or memory on processor 502.

The high speed controller 508 manages bandwidth-intensive operations forthe computing device 500, while the low speed controller 512 manageslower bandwidth-intensive operations. Such allocation of duties isillustrative only. In one implementation, the high-speed controller 508is coupled to memory 504, display 516 (e.g., through a graphicsprocessor or accelerator), and to high-speed expansion ports 510, whichmay accept various expansion cards (not shown). In the implementation,low-speed controller 512 is coupled to storage device 506 and low-speedexpansion port 514. The low-speed expansion port, which may includevarious communication ports (e.g., USB, Bluetooth, Ethernet, wirelessEthernet) may be coupled to one or more input/output devices, such as akeyboard, a pointing device, a scanner, or a networking device such as aswitch or router, e.g., through a network adapter.

The computing device 500 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 520, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 524. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 522. Alternatively, components from computing device 500 may becombined with other components in a mobile device (not shown), such asdevice 550. Each of such devices may contain one or more of computingdevice 500, 550, and an entire system may be made up of multiplecomputing devices 500, 550 communicating with each other.

Computing device 550 includes a processor 552, memory 564, aninput/output device such as a display 554, a communication interface566, and a transceiver 568, among other components. The device 550 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 550, 552,564, 554, 566, and 568, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 552 can process instructions for execution within thecomputing device 550, including instructions stored in the memory 564.The processor may also include separate analog and digital processors.The processor may provide, for example, for coordination of the othercomponents of the device 550, such as control of user interfaces,applications run by device 550, and wireless communication by device550.

Processor 552 may communicate with a user through control interface 558and display interface 556 coupled to a display 554. The display 554 maybe, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 556 may compriseappropriate circuitry for driving the display 554 to present graphicaland other information to a user. The control interface 558 may receivecommands from a user and convert them for submission to the processor552. In addition, an external interface 562 may be provide incommunication with processor 552, so as to enable near areacommunication of device 550 with other devices. External interface 562may provide, for example, for wired communication (e.g., via a dockingprocedure) or for wireless communication (e.g., via Bluetooth or othersuch technologies).

The memory 564 stores information within the computing device 550. Inone implementation, the memory 564 is a computer-readable medium. In oneimplementation, the memory 564 is a volatile memory unit or units. Inanother implementation, the memory 564 is a non-volatile memory unit orunits. Expansion memory 574 may also be provided and connected to device550 through expansion interface 572, which may include, for example, aSIMM card interface. Such expansion memory 574 may provide extra storagespace for device 550, or may also store applications or otherinformation for device 550. Specifically, expansion memory 574 mayinclude instructions to carry out or supplement the processes describedabove, and may include secure information also. Thus, for example,expansion memory 574 may be provide as a security module for device 550,and may be programmed with instructions that permit secure use of device550. In addition, secure applications may be provided via the SIMMcards, along with additional information, such as placing identifyinginformation on the SIMM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 564, expansionmemory 574, or memory on processor 552.

Device 550 may communicate wirelessly through communication interface566, which may include digital signal processing circuitry wherenecessary. Communication interface 566 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 568. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS receiver module 570 may provide additional wireless datato device 550, which may be used as appropriate by applications runningon device 550.

Device 550 may also communication audibly using audio codec 560, whichmay receive spoken information from a user and convert it to usabledigital information. Audio codex 560 may likewise generate audible soundfor a user, such as through a speaker, e.g., in a handset of device 550.Such sound may include sound from voice telephone calls, may includerecorded sound (e.g., voice messages, music files, etc.) and may alsoinclude sound generated by applications operating on device 550.

The computing device 550 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 580. It may also be implemented as part of asmartphone 582, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back-end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front-end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back-end, middleware, orfront-end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments of the invention have been described.Nevertheless, it will be understood that various modifications may bemade without departing from the spirit and scope of the invention. Forexample, various forms of the flows shown above may be used, with stepsre-ordered, added, or removed. Also, although several applications ofthe payment systems and methods have been described, it should berecognized that numerous other applications are contemplated.Accordingly, other embodiments are within the scope of the followingclaims.

1. (canceled)
 2. A method comprising: enabling, by one or morecomputers, selection of a matching criterion by a content sponsor thatprovides a creative, including presenting an interface to the contentsponsor that includes a control for selecting, from a continuous rangeof matching criteria, the given matching criterion; presenting, byincluding in the interface, performance information indicating ahistorical performance of the creative when presented according to eachof the matching criteria, including changing the performance informationpresented as the control is moved along the continuous range of matchingcriteria to present the performance information for a particularmatching criterion corresponding to the location of the control on thecontinuous range of matching criterion; and receiving, through theinterface, a selection of the given matching criterion from among thecontinuous range of matching criteria, wherein the selected givenmatching criterion has a corresponding historical performance;establishing different bid modifiers for a single instance of theselection keyword, the different bid modifiers being respectivelyassociated with different levels in the continuous range of matchingcriteria; identifying at least one request keyword; determining, by theone or more computers and based on a comparison of the at least onerequest keyword and the selection keyword, a given level of matchbetween the received at least one request keyword and the selectionkeyword; in response to determining the given level of match,identifying, based on the given level of match, that the given matchingcriterion is satisfied by the given level of match meeting thecorresponding matching threshold of the given matching criterion;identifying, from among the different bid modifiers for the givenmatching criterion, a given bid modifier that was established for thegiven matching criterion that is satisfied by the given level of match;and evaluating the creative for distribution based on an adjusted biddetermined based on application of the given bid modifier to acorresponding bid.
 3. The method of claim 2, further comprisingpresenting a matching criterion suggestion for a content sponsorassociated with the creative.
 4. The method of claim 2, wherein thecontrol is a sliding scale control.
 5. The method of claim 2, whereinestablishing the different bid modifiers includes automaticallyselecting the bid modifiers for a content sponsor associated with thecreative.
 6. The method of claim 2, wherein establishing the differentbid modifiers further includes enabling selection of the different bidmodifiers by a content sponsor associated with the creative.
 7. Themethod of claim 6, wherein enabling selection of the different bidmodifiers includes presenting an interface to the content sponsorassociated with the creative, the interface including a bid modifiercontrol that enables selection of different bid modifiers for differentportions of the range.
 8. The method of claim 7, wherein the bidmodifier control enables selection of a function to define the differentbid modifiers over the range.
 9. One or more non-transitorycomputer-readable media comprising instructions that, when executed byone or more processors, cause the one or more processors to performoperations comprising: enabling selection of a matching criterion by acontent sponsor that provides a creative, including presenting aninterface to the content sponsor that includes a control for selecting,from a continuous range of matching criteria, the given matchingcriterion; presenting, by including in the interface, performanceinformation indicating a historical performance of the creative whenpresented according to each of the matching criteria, including changingthe performance information presented as the control is moved along thecontinuous range of matching criteria to present the performanceinformation for a particular matching criterion corresponding to thelocation of the control on the continuous range of matching criterion;and receiving, through the interface, a selection of the given matchingcriterion from among the continuous range of matching criteria, whereinthe selected given matching criterion has a corresponding historicalperformance; establishing different bid modifiers for a single instanceof the selection keyword, the different bid modifiers being respectivelyassociated with different levels in the continuous range of matchingcriteria; identifying at least one request keyword; determining, basedon a comparison of the at least one request keyword and the selectionkeyword, a given level of match between the received at least onerequest keyword and the selection keyword; in response to determiningthe given level of match, identifying, based on the given level ofmatch, that the given matching criterion is satisfied by the given levelof match meeting the corresponding matching threshold of the givenmatching criterion; identifying, from among the different bid modifiersfor the given matching criterion, a given bid modifier that wasestablished for the given matching criterion that is satisfied by thegiven level of match; and evaluating the creative for distribution basedon an adjusted bid determined based on application of the given bidmodifier to a corresponding bid.
 10. The computer-readable media ofclaim 9, wherein the instructions cause the one or more processors toperform operations further comprising presenting a matching criterionsuggestion for a content sponsor associated with the creative.
 11. Thecomputer-readable media of claim 9, wherein the control is a slidingscale control.
 12. The computer-readable media of claim 9, whereinestablishing the different bid modifiers includes automaticallyselecting the bid modifiers for a content sponsor associated with thecreative.
 13. The computer-readable media of claim 9, whereinestablishing the different bid modifiers further includes enablingselection of the different bid modifiers by a content sponsor associatedwith the creative.
 14. The computer-readable media of claim 13, whereinenabling selection of the different bid modifiers includes presenting aninterface to the content sponsor associated with the creative, theinterface including a bid modifier control that enables selection ofdifferent bid modifiers for different portions of the range.
 15. Thecomputer-readable media of claim 14, wherein the bid modifier controlenables selection of a function to define the different bid modifiersover the range.
 16. A system comprising: one or more processors; and oneor more memory elements including instructions that when executed causethe one or more processors to perform operations comprising: enablingselection of a matching criterion by a content sponsor that provides acreative, including presenting an interface to the content sponsor thatincludes a control for selecting, from a continuous range of matchingcriteria, the given matching criterion; presenting, by including in theinterface, performance information indicating a historical performanceof the creative when presented according to each of the matchingcriteria, including changing the performance information presented asthe control is moved along the continuous range of matching criteria topresent the performance information for a particular matching criterioncorresponding to the location of the control on the continuous range ofmatching criterion; and receiving, through the interface, a selection ofthe given matching criterion from among the continuous range of matchingcriteria, wherein the selected given matching criterion has acorresponding historical performance; establishing different bidmodifiers for a single instance of the selection keyword, the differentbid modifiers being respectively associated with different levels in thecontinuous range of matching criteria; identifying at least one requestkeyword; determining, based on a comparison of the at least one requestkeyword and the selection keyword, a given level of match between thereceived at least one request keyword and the selection keyword; inresponse to determining the given level of match, identifying, based onthe given level of match, that the given matching criterion is satisfiedby the given level of match meeting the corresponding matching thresholdof the given matching criterion; identifying, from among the differentbid modifiers for the given matching criterion, a given bid modifierthat was established for the given matching criterion that is satisfiedby the given level of match; and evaluating the creative fordistribution based on an adjusted bid determined based on application ofthe given bid modifier to a corresponding bid.
 17. The system of claim16, wherein the instructions cause the one or more processors to performoperations further comprising presenting a matching criterion suggestionfor a content sponsor associated with the creative.
 18. The system ofclaim 16, wherein the control is a sliding scale control.
 19. The systemof claim 16, wherein establishing the different bid modifiers includesautomatically selecting the bid modifiers for a content sponsorassociated with the creative.
 20. The system of claim 16, whereinestablishing the different bid modifiers further includes enablingselection of the different bid modifiers by a content sponsor associatedwith the creative.
 21. The system of claim 20, wherein enablingselection of the different bid modifiers includes presenting aninterface to the content sponsor associated with the creative, theinterface including a bid modifier control that enables selection ofdifferent bid modifiers for different portions of the range.
 22. Thesystem of claim 21, wherein the bid modifier control enables selectionof a function to define the different bid modifiers over the range.